Transferring knowledge as heuristics in reinforcement learning: A case-based approach

The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain. A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms. © 2015 Elsevier B.V.Luiz Celiberto Jr. and Reinaldo Bianchi acknowledge the support of FAPESP (grants 2012/14010-5 and 2011/19280-8). Paulo E. Santos acknowledges support from FAPESP (grant 2012/04089-3) and CNPq (grant PQ2 -303331/2011-9).Peer Reviewe

mRNA Display Selection of an Optimized MDM2-Binding Peptide That Potently Inhibits MDM2-p53 Interaction

p53 is a tumor suppressor protein that prevents tumorigenesis through cell cycle arrest or apoptosis of cells in response to cellular stress such as DNA damage. Because the oncoprotein MDM2 interacts with p53 and inhibits its activity, MDM2-p53 interaction has been a major target for the development of anticancer drugs. While previous studies have used phage display to identify peptides (such as DI) that inhibit the MDM2-p53 interaction, these peptides were not sufficiently optimized because the size of the phage-displayed random peptide libraries did not cover all of the possible sequences. In this study, we performed selection of MDM2-binding peptides from large random peptide libraries in two stages using mRNA display. We identified an optimal peptide named MIP that inhibited the MDM2-p53 and MDMX-p53 interactions 29- and 13-fold more effectively than DI, respectively. Expression of MIP fused to the thioredoxin scaffold protein in living cells by adenovirus caused stabilization of p53 through its interaction with MDM2, resulting in activation of the p53 pathway. Furthermore, expression of MIP also inhibited tumor cell proliferation in a p53-dependent manner more potently than DI. These results show that two-stage, mRNA-displayed peptide selection is useful for the rapid identification of potent peptides that target oncoproteins

The genomes of two key bumblebee species with primitive eusocial organization

Background: The shift from solitary to social behavior is one of the major evolutionary transitions. Primitively eusocial bumblebees are uniquely placed to illuminate the evolution of highly eusocial insect societies. Bumblebees are also invaluable natural and agricultural pollinators, and there is widespread concern over recent population declines in some species. High-quality genomic data will inform key aspects of bumblebee biology, including susceptibility to implicated population viability threats. Results: We report the high quality draft genome sequences of Bombus terrestris and Bombus impatiens, two ecologically dominant bumblebees and widely utilized study species. Comparing these new genomes to those of the highly eusocial honeybee Apis mellifera and other Hymenoptera, we identify deeply conserved similarities, as well as novelties key to the biology of these organisms. Some honeybee genome features thought to underpin advanced eusociality are also present in bumblebees, indicating an earlier evolution in the bee lineage. Xenobiotic detoxification and immune genes are similarly depauperate in bumblebees and honeybees, and multiple categories of genes linked to social organization, including development and behavior, show high conservation. Key differences identified include a bias in bumblebee chemoreception towards gustation from olfaction, and striking differences in microRNAs, potentially responsible for gene regulation underlying social and other traits. Conclusions: These two bumblebee genomes provide a foundation for post-genomic research on these key pollinators and insect societies. Overall, gene repertoires suggest that the route to advanced eusociality in bees was mediated by many small changes in many genes and processes, and not by notable expansion or depauperation

Unveiling thermal transitions of polymers in subnanometre pores

The thermal transitions of confined polymers are important for the application of polymers in molecular scale devices and advanced nanotechnology. However, thermal transitions of ultrathin polymer assemblies confined in subnanometre spaces are poorly understood. In this study, we show that incorporation of polyethylene glycol (PEG) into nanochannels of porous coordination polymers (PCPs) enabled observation of thermal transitions of the chain assemblies by differential scanning calorimetry. The pore size and surface functionality of PCPs can be tailored to study the transition behaviour of confined polymers. The transition temperature of PEG in PCPs was determined by manipulating the pore size and the pore–polymer interactions. It is also striking that the transition temperature of the confined PEG decreased as the molecular weight of PEG increased

Should patients with hip joint prosthesis receive antibiotic prophylaxis before dental treatment?

The safety committee of the American Academy of Orthopedic Surgeons (AAOS) recommended in 2009 that clinicians should consider antibiotic prophylaxis for all patients with total joint replacement before any invasive procedure that may cause bacteremia. This has aroused confusion and anger among dentists asking for the evidence. The present review deals with different aspects of the rationale for this recommendation giving attention to views both in favor of and against it

Group II Intron-Based Gene Targeting Reactions in Eukaryotes

Mobile group II introns insert site-specifically into DNA target sites by a mechanism termed retrohoming in which the excised intron RNA reverse splices into a DNA strand and is reverse transcribed by the intron-encoded protein. Retrohoming is mediated by a ribonucleoprotein particle that contains the intron-encoded protein and excised intron RNA, with target specificity determined largely by base pairing of the intron RNA to the DNA target sequence. This feature enabled the development of mobile group II introns into bacterial gene targeting vectors ("targetrons") with programmable target specificity. Thus far, however, efficient group II intron-based gene targeting reactions have not been demonstrated in eukaryotes.By using a plasmid-based Xenopus laevis oocyte microinjection assay, we show that group II intron RNPs can integrate efficiently into target DNAs in a eukaryotic nucleus, but the reaction is limited by low Mg(2+) concentrations. By supplying additional Mg(2+), site-specific integration occurs in up to 38% of plasmid target sites. The integration products isolated from X. laevis nuclei are sensitive to restriction enzymes specific for double-stranded DNA, indicating second-strand synthesis via host enzymes. We also show that group II intron RNPs containing either lariat or linear intron RNA can introduce a double-strand break into a plasmid target site, thereby stimulating homologous recombination with a co-transformed DNA fragment at frequencies up to 4.8% of target sites. Chromatinization of the target DNA inhibits both types of targeting reactions, presumably by impeding RNP access. However, by using similar RNP microinjection methods, we show efficient Mg(2+)-dependent group II intron integration into plasmid target sites in zebrafish (Danio rerio) embryos and into plasmid and chromosomal target sites in Drosophila melanogster embryos, indicating that DNA replication can mitigate effects of chromatinization.Our results provide an experimental foundation for the development of group II intron-based gene targeting methods for higher organisms

The Herschel-SPIRE Legacy Survey (HSLS): the scientific goals of a shallow and wide submillimeter imaging survey with SPIRE

A large sub-mm survey with Herschel will enable many exciting science opportunities, especially in an era of wide-field optical and radio surveys and high resolution cosmic microwave background experiments. The Herschel-SPIRE Legacy Survey (HSLS), will lead to imaging data over 4000 sq. degrees at 250, 350, and 500 micron. Major Goals of HSLS are: (a) produce a catalog of 2.5 to 3 million galaxies down to 26, 27 and 33 mJy (50% completeness; 5 sigma confusion noise) at 250, 350 and 500 micron, respectively, in the southern hemisphere (3000 sq. degrees) and in an equatorial strip (1000 sq. degrees), areas which have extensive multi-wavelength coverage and are easily accessible from ALMA. Two thirds of the of the sources are expected to be at z > 1, one third at z > 2 and about a 1000 at z > 5. (b) Remove point source confusion in secondary anisotropy studies with Planck and ground-based CMB data. (c) Find at least 1200 strongly lensed bright sub-mm sources leading to a 2% test of general relativity. (d) Identify 200 proto-cluster regions at z of 2 and perform an unbiased study of the environmental dependence of star formation. (e) Perform an unbiased survey for star formation and dust at high Galactic latitude and make a census of debris disks and dust around AGB stars and white dwarfs